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ABSTRACT 

In its second extension year following a 3-year 
federal funding cycle, Project GET SET served 163 junior high school 
students of limited ability in aoth English and Spanish at two Bronx 
junior high schools. The project's _im was to reinforce English and 
native language skills and to offer students personal counseling, 
informative programs that would involve their parents, and career 
guidance, students also worked with a part-time tutor. The program 
met its proposed objectives for native language development, and the 
noninstructional objectives were met for career seminars, parent 
programs, and staff training and development. Project GET SET only 
partially achieved the objectives in Englsih language development and 
curriculum material preparation. The project exposed students to a 
var^aty of traditional and nontraditional career opportunities 
through a range of bilingual reference works, field trips, and 
lecture programs. Parents could participate in four workshops, and 
teachers and guidance counselors each attended three workshops. 
Recommendations for improvement include completion of the educational 
curriculum materials, and training mainstream teachers in the 
project 5 s instructional methods. (MSE) 
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PROJECT GET SET* 
1S37-88 

SUMMARY 



Project GET SET was fully implemented. During the 
1987-88 year the program provided English- and native- 
language instruction and a career awareness curriculum 
to 163 students. It also provided training programs 
for parents and for school staff. 

Project students- partially met the proposed objective 
for English-language development and met the objective 
for native-language development. The project met its 
noninstructional objectives in the areas of 
extracurricular activities for students, parental 
training, and staff development, and partially met the 
objective for materials development. 

In its second extension year following a three-year Title 
VII funding cycle, Project GET SET served 163 junior high school 
students of limited ability in both English and Spanish. The 
students attended J.H.S. 80 and J.H.S. 143, both located in 
economically depressed, predominantly Hispanic neighborhoods in 
the Bronx. 

Project GET SET's aim was to reinforce English- and native- 
language skills; to offer personal counseling to students and 
informative programs that would involve their parents; and to 
offer career guidance to students. The project was administered 
by the Office of Bilingual Education (O.3.E.) of the New York 
City Public Schools. 

A resource specialist provided participating students with 
supplementary instruction in English and career skills, 
coordinated project activities and resources with the director, 
helped with administrative tasks and program implementation, and 
instructed te^jhers in the use of project-supplied curricula and 
curricular aids. GET SET organized career-information seminars 
for students, and a variety of informative programs for parents. 
Students also worked with a part-time student tutor. Project 
personnel used the Language Assessment Battery (LAB), English and 
Spanish versions, and the Degrees of Reading Power (D.R.P.) test 
in pretesting and posttesting. 

The Office of Research, Evaluation, and Assessment (OREA) 
examined student performance on the LAB and D.R.P. tests, 
reviewed program materials, and evaluated information obtained 



*This summary is based on the final evaluation of the "Project 
GET SET 1987-88" prepared by the OREA Bilingual Education 
Evaluation Unit. 



from interviews of program and school personnel and from 
classroom observation in their evaluation of the program. 

Project GET SET met its proposed objective for native- 
language development and its noninstructional objective 
concerning student career seminars, programs for parents, and 
staff training and development. The program did not meet its 
objective for English-language development and partially achieved 
its objective in the preparation of curriculum materials. 

Students' gains in English-language proficiency for 1987-88 
were consistent with those observed in 1986-87. in 1986-87 
students tested with the D.R.P. showed small gains in English 
proficiency. This year, students showed no significant 
improvement when tested with the D.R.P. , but pre- to posttest 
scores on the LAB evidenced significant improvement in English- 
language skills. During the current program year, students 
showed even greater gains in native-language development than 
they had last year. 

Project GET SET exposed students to a variety of traditional 
and nontraditional career opportunities through a range of 
bilingual reference works, field trips, and lecture programs. 
Parents could participate in four workshops; teachers and 
guidance counselors attended three workshops each. The project 
developed an information packet. 

The conclusions, based on the findings of this evaluation 
lead to the following recommendations: ' 

• The project should complete the educational curriculum 
materials. 

• Mainstream staff and teachers should receive full 
training in GET SET's instructional methods so that 
they can incorporate these elements into the standard 
curriculum, establishing continuity in students' 
language development and exposure to a range of career 
opportunities. 
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I. INTRODUCTION 



The 1987-88 year of Project GET SET was the second extension 
year following a three-year Title VII f ui iing cycle. Since its 
inception five years ago, Project GET SE1 has provided supportive 
and peer-tutorial services to dually limited* Hispanic junior 
high school students, and pursued the following objectives: to 
reinforce English-language skills, especially reading and 
writing; to reinforce native language skills; to offer personal 
counseling services; to offer career guidance; and to instruct 
teachers and other staff in this curriculum. In this final year, 
three program materials for establishing future curricula were to 
be developed. 

In 1987-88, GET SET served 163 limited English proficient 
(LEP) students in grades seven through nine. About half of the 
students were in eighth grade. Most of the students were born in 
the United States, the Dominican Republic, and Puerto Rico (see 
Table 1) . 

The overwhelming majority of project students came from 
family backgrounds characterized by low literacy levels (in both 
Spanish and English), poverty, and often emotional problems. 
Their educational deficits and difficult family circumstances put 
them at severe risk of failing academically and dropping out of 
school . 



^ ^Students scoring below the twenty-first percentile on the 
English version of the Language Assessment Battery (LAB) and 
even lower on the Spanish version. 



ERLC 



,10 



TABLE 1 



Number and Percent of Project Students 
by Country of Birth 





Country of Birth 


Niuober 


Percent 



United States 


79 


49.1 


Dominican Republic 


43 


26.7 


Puerto Rico 


24 


14.9 


Panama 


3 


2.0 


Cuba 


2 


1.2 


Mexico 


2 


1.2 


El Salvador 


2 


1.2 


Honduras 


1 


0.6 


Colombia 


1 


0.6 


Ecuador 


1 


0.6 


Peru 


1 


0.6 


Bolivia 


1 


0.6 


Central and South America 






(Country Unspecified) 


1 


0.6 


TOTAL 


161 8 


99. 9 b 



a Data were missing for two students. 

lumbers do not sum to 100 because of rounding. 

•Most participants were born in the United States. 



DELIVERY OF SERVICES 

It was necessary to plan and modify project activities for 
this final year of funding in order to ensure that some degree of 
institutionalization and information dissemination among the 
teaching and guidance staff would take place. 

A student tutor from James Monroe High School aided the 
resource specialist, who worked with small groups of students to 
develop their language skills, while exposing them to career- 
oriented issues. 

This year the project offered four parent workshops. Topics 
included information for parents about services provided by 
ASPIRA". The workshops were offered at both sites; attendance 
varied from 10 to 14 persons. 

The project offered three workshops to Community School 
District (C.S.D.) lO's guidance staff in order to inform them 
about the project's services, to discuss differences that may 
occur when dealing with culturally diverse students, and to 
inform counselors about ASPIRA's services. Classroom teachers 
also received services from Project GET SET. There were three 
in-service training opportunities where teachers might increase 
their knowledge of resources, their understanding of LEP 
students, and their utilization of community and field resources 
in the development of students 1 language ability. The project 
director designed and distributed a newly formatted booklet that 

*ASPIRA is ci nonprofit national organization with local 
chapters, dedicated to promoting the educational advancement of 
Hispanic youth. 

3 



described the project and its services, and included sample 
lesson plans. 

Students from several Bronx junior high and intermediate 
schools attended an all-day career fair and special assembly held 
at Lehman College. 

STAFF 

GET SET's staff consisted of a project director, a resource 
specialist, and a part-time student tutor. The project director 
provided the resource specialist with guidance and practical 
support, reviewed assessment instruments, furnished evaluation 
information, and managed the day-to-day activities of the 
program. 

The resource specialist spent two days a week at one site 
and three days a week at the other. She provided participating 
students with supplementary instruction in English and career 
skills, helped with program implementation, and instructed 
teachers in the use of project-supplied curricula and curricular 
aids. 

The project director for the 1987-88 year had earned a 
doctorate in curriculum/instruction, and had considerable 
background experience as a teacher, trainer, and researcher. 

The recipient of a master* s degree, the resource specialist 
had many years of experience as a community worker and career and 
guidance counselor for minor5.ties, in particular, for Hispanic 
women. She had been a bilingual teacher for seven years. 
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SETTING 

Both sites were located in economically depressed parts of 
C.S.D. 10 in the Bronx; the majority of students were eligible 
for the federally sponsored free lunch program. Both schools 
were overcrowded. 

HISTORY OF PROGRAM 

This was the project's fifth year. A full description of 
the previous years can be found in the evaluation report of 1986- 
87. 

REPORT FORMAT 

This report is organized as follows: Chapter II describes 
the evaluation methodology; Chapter III presents an analysis of 
the qualitative and quantitative findings of the evaluation; and 
Chapter IV offers conclusions and recommendations based on the 
results of the evaluation. 



II. EVALUATION METHODOLOGY 



EVALUATION QUESTIONS 

The evaluation assessed two major areas: program 
implementation and outcomes. Evaluation questions are as 
follows : 

Process/Implementation 

• Have the instructional activities for developing English- 
language proficiency been implemented as described? 

• Have the instructional activities for developing native- 
language proficiency been implemented as described? 

• Have the students received personal counseling? 

• Has the project provided career education and counseling? 

• Did the project hold at least three workshops for parents? 
Outcome 

• Whst was the average gain in English-language proficiency 
and English reading skills, as measured by the LAB and 
D.R.P.? 

• What was the average gain in native-language proficiency 
and test scopes, as measured by the Spanish version of the 
LAB? 

EVALUATION PROCEDURES 
Sample 

The Office of Research, Evaluation, and Assessment {OREA) 
evaluated Project GET SET through on-site visits to scheduled 
events, interviews with project staff; and an examination of 
documents, project products, and workshop evaluations. It also 
examined student achievement data. 

6 
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Instruments 

To evaluate students' progress in English-language 
development, students were pretested and posttested on the 
Degrees of Reading Power (D.R.P.) Test," 

To evaluate students' progress in native- language 
development, students were pretested and posttested on the 
Spanish version of the Language Assessment Battery * » (LAB). The 
project also considered other indicators, such as report card 
grades. 

Data Analysi s 

In order to assess achievement, raw scores on the D.R.P. and 
LAB were converted to normal curve equivalent (N.C.E.) scores.*** 



* The Degrees of Reading Power Test was developed by the 
College Board to provide information about student reading 
ability on the same scale used to describe the difficulty of 
textbooks . 



** T he Language Assessment Battery (LAB) was developed by 
the Board of Education of the City of New York to measure the 
English-language proficiency of nonnative speakers of English in 
order to determine whether their level of English proficiency is 
sufficient to enable them to participate effectively in classes 
taught in English. Students scoring below the twenty-first 
percentile on the LAB are entitled to bilingual and E.S.L. 
services. 

***Raw scores were converted to N.C.E. scores, which are 
normalized standard scores. They have the advantage of forming 
an equal interval scale so that scores and gains can be averaged. 
Forthe norming population, N.C.E.s have a mean of 50, a standard 
deviation of approximately 20, and a range from one to 99. 
Scores can be compared to the norming population. 

7 
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Correlated t-tests and effect sizes* on pretest and posttest 
N.C.E. scores for each grade level were computed to determine if 
the changes in scores were statistically significant. 

Li mitations 

Since all program-eligible students were involved either in 
the current project or another program, it was impossible to 
select a similar comparison group. In addition, since English 
proficient students do not take pretests and posttests of the 
LAB, a gap-reduction design was not feasible. 

The project did not provide data on all program 
participants. However, the number of students for whom there 
were data were large enough to make the analyses meaningful. 



*The effect size, developed by Jacob Cohen, is a ratio of 
the mean gain to the standard deviation of the gain. This ratio 
provides an index of improvement in standard deviation units 
irrespective of the size of the sample. Effect size (E.S.) is 
interpreted to indicate educational meaningfulness, and an E.S. 
of .80 is thought to be highly meaningful, while one of .2 is 
considered to be only slightly so. 

8 
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III. EVALUATION FINDINGS 



INSTRUCTIONAL ACTIVITIES 

Project GET SET provided English- and native-language 
instruction; enrichment classes on career options and skills; 
personal and career counseling; and parent, teacher, and guidance 
counselor workshops. 

English as a Second Language 

The program objective for English as a second language was 
as follows: 

By June 1988, participating students will demonstrate a 
significant increase in English language skills as 
measured by the Language Assessment Battery and/or the 
Degrees of Reading Power Test (D.R.P.). 

The resource specialist provided students with supplementary 
instruction in English, with the aid of the student tutor, she 
worked with groups of five to 15 students to develop their 
language skills through exposure to career-oriented issues. 

Achievement in English as a Second Language . Students were 
pretested and posttested on the D.R.P. In order to assess 
achievement, raw scores were converted to normal curve equivalent 
(N.C.E.) scores. Correlated t-tests on pretest and posttest 
N.C.E. scores for each grade level were computed to determine if 
the changes in scores were statistically significant. (See Table 
2.) 

The project provided data for 122 students~73 percent of 
the target group, of this number, 47 had complete results for 
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TABLE 2 



Pretest/Postest N.C.E. Differences 
on the Language Assessment Battery , by Grade 



Grade 


N 


Pretest 


Posttest 


Difference 


t 

Value 


Effect 
Size 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


7 


16 


10.5 


9.0 


11.9 


10.1 


1.5 


7.3 


.80 


.21 


8 


17 


8.6 


9.6 


14.8 


11.9 


6.2 


5.7 


4.35* 


1.09 


9 


14 


9.8 


9.2 


18.7 


12.7 


8.9 


6.5 


4.94* 


1.37 


TOTAL 


47 


9.6 


9.1 


5.0 


11.6 


5.3 


7.1 


5.16* 


.74 



* p<.05 

• Eighth and ninth graders made statistically significant gains 
on the LAB. 
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the LAB, and 75 had complete results for the D.R.P. Students 
tested with the D.R.P. did not show any significant increase in 
their scores; however, eighth- and ninth-grade students tested 
with the LAB did show a significant increase (p<.05) from pre- to 
posttest. These students made mean gains of 6.2 and 8.9 N.C.E.s, 
respectively. The corresponding effect sizes of 1.09 and 1.37, 
respectively, indicate that the increase in scores was highly 
educationally meaningful. The change in scores of seventh-grade 
students showed a small effect size (0.21). For them the 
increase was not educationally meaningful. Overall, the students 
exhibited a mean N.C.E. gain of 5.3 (effect size = .74) and this 
proved to be significant. Therefore, this objective was 
achieved. 

Native Language Arts 

The program objective for native language arts was as 
follows: 

By June 1988, participating students will demonstrate a 
significant increase in Spanish language arts skills as 
measured by the Language Assessment Battery (LAB) 
and/or other indicators such as report card grades, 
etc. 

The resource specialist provided students with instruction 
in Spanish, using a text that emphasizes writing skills and 
focuses on themes related to immigration. Students developed 
skills in Spanish in conjunction with the career-awareness 
curriculum, which stressed the importance of bilingual skills. 
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Achievement in Native Language Arts , Correlated £-tests on 
pretest and posttest N.C.E. scores for each grade level were 
computed to determine if the changes in scores were 
statistically significant. (See Table 3.) Project GET SET 
provided data for 118 students. Students in all three grades 
made significant gains (p<.05) in their N.C.E. scores. The 
large effect sizes corresponding to these N.C.E. gains bolster 
the finding that these gains were educationally meaningful. The 
program achieved its proposed objective in this area. 
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TABLE 3 



Pretest/Posttest N.C.E. Score Differences 
on the Language Assessment Battery (Spanish Version) , by Grade 



Grade 


N 


Pretest 


Posttest 


Difference 


T 

Value 


Effect 
Size 


Mean 


S.D. 


Mean' 


S.D. 


Mean 


S.D. 


7 


34 


50.5 


2.8 


54.9 


6.5 


4.4 


4.3 


5.92* 


1.02 


8 


52 


51.0 


2.8 


56.6 


6.2 


5.6 


4.0 


9.81* 


1.40 


9 


32 


52.5 


4.1 


59.9 


7.9 


7.4 


4.3 


9.57* 


1.72 


TOTAL 


118 


51.3 


3.3 


57.0 


7.0 


5.7 


4.3 


14.34* 


1.33 



* p<.05 

• Students at all grades made statistically significant gains. 



Career-Related Instruction 

The program objective for career-related instruction was as 
follows: 

By June 1988, participating students will demonstrate 
a^ higher level of career awareness and knowledge of 
bilingual and nontraditional career opportunities, as 
compared with non-project students, as measured by 
project-developed tests. 

The project set up a career enrichment center where 
students could consult Spanish- and English-language books on a 
wide range of careers. The center also maintained a large 
collection of employment-related films in both languages, and 
GET SET organized regular screenings of these films at both 
sites. Additionally, the resource specialist was developing 
bilingual curricula covering a broad range of careers. 

Achievement in Career-Related Instruction . Although the 
project had sample tests in English and Spanish, the project did 
not provide data to assess this objective. 

NONINSTRUCTIONAL ACTIVITIES 

Field Trips and Guest Speakers 

The program objective for field trips and guest speakers 

was as follows: 

By June 1988, Project GET SET will have provided at 
least four field trips and four guest speakers related 
to bilingual careers and planning for high school, as 
documented by project records. 

Project GET SET staff organized field trips to enhance 

students 1 awareness of and interest in a wide range of career 

options. Among the sites visited were Roberto Clemente State 
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Park, Capital National Bank, Hostcs College, Columbia 
University, and Lehman College, where they participated in a 
day-long bilingual career fair. Several Hispanic guest 
lecturers addressed the classes. The project met its objective. 

Staff Development 

The program objectives for staff development were as 
follows: 

By June 1988, Project GET SET will have offered at 
least three on-site training workshops on E.S.L., 
bilingual career education resources, career education 
materials, etc., for participating project teachers, 
as documented by project records. 

By June 1988, Project GET SET will have sponsored at 
least three workshops for district guidance 
counselors, on multicultural counseling, community 
resources, and career education needs of Hispanic 
students and families, as documented by project 
records. 

Project GET SET offered three workshops for teachers at one 
of the program sites. The workshops covered the following 
topics: an overview of services offered to GET SET students, how 
to better understand LEP students, and how to use field trips 
and community resources as catalysts for language development. 
Thus, the program achieved the first objective in this area. 

The project offered guidance personnel in C.S.D. 10 three 
workshops related to students with limited ability in both 
English and their native language: two dealing with 
multicultural issues within the counseling setting, and a third 
highlighting the offerings of ASPIRA. Thus, rroject GET SET 
achieved the second objective in this area. 

15 
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Parental Training 

The program objective for parental training was as follows: 

By June 1988, Project GET SET will have offered at 
least tnree parent workshops on bilingual career 
education and planning for high school/college, as 
documented by project records. 

Parents attended a number of sponsored workshops on such 

topics as "Teenagers and Their Health" and on the programs and 

services of ASPIRA. The project met this objective. 



Curriculum and Materials Development 

The program objective for curriculum and materials 

development was as follows: 

By August 1988/ Project GET SET will have developed 
the following items: 

a. Project GET SET Information Packet 

b. Project GET SET Sample Lesson Plans for Career 
Education in Spanish and English 

c. Project GET SET (1987-88) Program Highlights 
Pamphlet 

According to the project director, the first and third 
proposed items were completed, but the remaining one — the sample 
lesson plans (item b.) — was in draft form. The program 
partially met this objective. 
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IV. CONCLUSIONS AND RECOMMENDATIONS 



Project GET SET successfully provided most of the services 
it had proposed. This was largely due to the dedication and 
commitment of the resource specialist and project director, as 
well as the cooperation of the principals at both sites. 

Operating out of enrichment centers located in small corners 
of the schools' libraries, the project staff provided 
supplementary instruction in English-language skills and native 
language arts, and offered career guidance. Through a program of 
extracurricular trips and guest lectures, the staff sought to 
foster students 1 self-confidence and cultural awareness, and to 
convey the value of bilingual skills and the necessity of a high 
school diploma in the contemporary job market. 

A review of student performance data revealed that Project 
GET SET achieved its instructional objectives only in E.S.L. a~ * 
native language arts. Finally, the project did not provide data 
to measure an increase in student awareness of career 
opportunities. 

The attempt to integrate the project's career and guidance 
material with the mainstream curriculum received the enthusiastic 
support of the administrate n of both schools, and the project 
held three workshops each for teachers and guidance counselors to 
familiarize them with the project's goals and methods. 

The project held workshops for parents on a variety of 
> career- and health-oriented topics, thus fulfilling the parental 

17 
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training objective. 

The project achieved its noninstructional objectives in 
extracurricular activities for students, parental training, and 
staff development. The objective in curriculum development was 
partially achieved: only two out of the three proposed items were 
completed. 

It is important to note that the project was still able to 
meet many of its objectives despite sharp cuts in its fiscal and 
staff resources, as well as four changes in project leadership. 
The final project director was committed to making frequent 
visits to the sites, and was actively involved in all aspects of 
the program. The entire project staff, given the resources 
available to them, worked together effectively to meet the needs 
of the dually limited student. 

The conclusions, based on the findings of this evaluation, 
lead to the following recommendations: 

• The project should complete the educational curriculum 
materials. 

• Mainstream staff and teachers should receive full 
training in GET SET's instructional methods so that 
they can incorporate these elements into the standard 
curriculum, establishing continuity in students' 
language development and exposure to a range of career 
opportunities . 
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